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ABSTRACT 

This paper presents & critical review of computer 
assisted instruction (CAI); an overview of recent intelligent 
tutoring systems (ITSs) , including current perceived shortcomings; 
mstjor activities of the field i.e., analysis of traching/lmarning 
processes, and extending and developing artificial intelligence 
techniques for use in intelligent tutoring systems; and a methodology 
for building ITSs. Examples are given from recent work on a student 
modelling system used to develop an ITS for algebra. A research 
agenda for ITS development relating to current activities in 
artificial intelligence <AI) and cognitive science is suggested and a 
list of 36 references is provided. (MBR) 
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The paper includes a critical review of CAI, a overview of recent ITSs 
(Intelligent Tutoring System) and a methodology for building ITSs. Examples are 
given fee. the author's recent work on a student modelling system. Further the 
paper suggests a research agenda for the sub-field and relates this to current 
activities in Al and cognitive science. 

1. Introduction 

The pioneers of CAI in the late 6Q's suggested that through this mediun, 
highly individualized instruction would become common place. Two decades' later we 
see fairly widespread use of computer-based instructional materials but very 
little of it could be said to be adaptive. Why this shortfall? I believe the key 
problem with .virtually all authoring languages including those available today, is 
that t,,e author of the teaching material has to provide in advance a list of 
anticipated responses (and associated actions). Thus such systems are only able 
to deal with situations which have been pre.pecif ied, and are. unable to respond 

appropriately to novel errors, or for that matter to brilliant insights. 

Building teaching systems which are truly adaptive is a very demanding task. 

A decade or so ago it was realised that this would not 
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be achieved without the use of artificial intelligence technique** Car bone 11 
[ 1970] - Subsequently Hartley end Sleeman (1973] suggested thet intelligent 
tutoring system* would normally have four distinct databases: 

. knowledge of the task domain 

. a model/history of the student's 
behavior 

a list of possible teaching operations 
e mean-ends guidance rules which relate 

teaching decisions to conditions in ^ 
the student model 

The very earliest systems to encorporate some of these databases where 
programs which generated tasks* For example Uhr and his collaborators 
implemented programs which generated simple arithmetic and vocabulary recall 
tasks, Uhr [1969]. Subsequently, systems were implemented which attempted to 
create a task which was appropriate to the student's competence in the task 
domain, Suppes { 1969 J and Woods and Hartley [1971]. These adaptive systems 
included the four data-bases given earlier but often in a very simplistic form. 
The initial version of the Leeds Adaptive arithmetic system, for instance, used a 
limited number of teaching operations and its student model consisted rerely of an 
integer to indicate the level of the student's competence. On the other hand the 
original scholar system, Carboneil [1970], used a recently introduced 
representation, namely a semantic net, for the systems' domain knowledge and the 
student model. Nodes in the network had tags associated with them to indicate 
whether the concept was, or was not, known to .the student. However, SCHOLAR- I had 
a poorly articulated teaching strategy, which was not represented as a separate 
data-base* 



4 



- 3 - 



2. Review of Recent ITSa 
In the interim a number of knowledgeable/ intelligent teaching systems have 
been implemented. Figure 1 lists many of .these systems and indicates their 
subject domains. Many bf these systems have been used to provide supportive 
problem solving and not initial instruction as this is felt to be a more 
appropriate use. A further general point is that currently ITSs are an order of 
magnitude -more expensive to implement than regular CAI (it is usually estimated 
that an hour of CAI material requires. 100 hours of an experienced author's time to 
prepare), and so it is important that the implementore of these systems choose 
their topics well. I believe the, topics listed in figure 1 are important ones. 
Indeed I believe that many of them are educational "watersheds", that is they 
represent, topics which if not mastered will render further progress in the field 
(virtually) impossible. ,For example, it is highly significant if a child fails 
to become competent at clerk's or vertical arithmetic. 
(Figure 1 about here] 



As nojed earlier .bui lding a truly responsive teaching systems implies 
solving a range of very open-ended problems. Each of the systems implemented has 
tended to emphasize some aspects and neglect c hers. In a recent overview Sleeraan 
and Brown [1982 J suggested that'eurrent perceived shortcomings include: 

1. The instructional material produced in response to a student's query or 
mistake is often at the wrong level of detail, as the system assumes, too much or 
fco^ little student knowledge. 

2. The system assumes a particular conceptualisation of the domain, thereby 
coercing a student's performance into its own conceptual framework. None of these 
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. tyste*. can discover, end work within, eh* indent*! on (idio.yncratic) 
conceptuali.etion to diagnose hi. "mind bug." within that framework, 

3. The tutoring and critiquing strategic, used by these «y. terns are 
^excessively ad hoc reflecting unprincipled intuition, about how to control their 

behaviour. Di.covering con.i.tent principle, would be facilitated by constructing 
better theorie. of learning and mi.le.rning - a task requiring detailed 
psychological theorie* of knowledge repre.entation and belief revision. 

4. User interaction is .till too restrictive, limiting the student's 
expressiveness and thereby limiting the ability of ' the tutor's diagnostic 
mechanisms* 

The field has subsequently concentrated most of its efforts on two major 
activities: 

1. Understanding the nature of learning, mi.-learning and teaching 
processes. 

2. Extending and developing . AI techniques for use in ITSs. 



2 .1. Analysis of teaching/ learning processes . If , does not have a gQQd M 
for the types of misunderstandings which occur with a particular subject domain, 
then it is impossible to write CAI material or implement ITSs which can deal wit 
the domain effectively. Collins and his group have done significant work on 
protocol analysis within the ITS community in the subject areas of geography and 
meterology, Stevens, Collins, and Coldin' { 1982]. More recently Mat* (1982] and 
Sleeman (in press] have analysed students' difficulties wi th beginning" algebra. 
However, workers in science education who have been studying the differences 
between experts and novices and those who work ^n the field of mental models have 
also contributed significantly to our knowledge of these issues, see for example , 
Stevens & Centner [1983], Davis, Jockusch & McKnight (1978J. 
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h2 AI Technique* evo lved by the ITS Community 

A. noted earlier there .re many central AI problems to be solved before 
powerful general purpo.e ITS. can be i.plemented. Before we review the techniques 
which have been implemented, I would like W note the additional stringent 
requirement, which are met in this area (and are now being encountered in other 
areas of applied AI like Expert Systems). Namely for the systems to be acceptable 
in the field they must be robust (that is they must not crumble when they 
encounter a response which is out of their range), they must be able to cope with 
responses which are both incomplete and inconsistent, and thirdly they must 
respond fairly quickly. The techniques to be highlighted here include: 

. Student modelling and concept , 
formation. 

. Friendly Natural Language Systems. 

2.2,1 Student Modelling 

A student model was one of the data-bases which Hartley and Sleeman [19 73] 
suggested should be part of each ITS. The issue which has ocen addressed more 
recently is that of inferring such models from observing student's performance. 
The principal issues addressed have been developing techniques to: 

. avoid the combinatorial explosion when producing models from primitive 
rules, 

• cope with noisy data 

Suppose one is attempting to model the incorrect behavior of students in a 
particular domain. The approach taken by the BUGGY [Brown & Burton, 1978] and LMS 
projects [Sleeman, 1982] is to provide the modelling system with a set of correct 
domain rules and associated incorrect rules which have been noted previously in 
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protocol.. If fro. t..k-.n.ly.i. one know. eh.t N (primitive) rule, .re nece...ry 
to .olve the t.,k, then . simple model-generating .lgorithn will produce N! 
v.ri.nts; thi. i. . prohibitively Urge number even for modest .i.ed rule bases. 
(And of cour.e including incorrect, or ,.1-rule. will merely incre... the number 
of model..) The BUGGY project overcame thi. problem by u.ing . set of heuri.tic. 
to determine the combinetion. of rule, which could occur in .ny on^model. The 
UiS.T.roject focu.ed on . particular rule for e.ch .et of t..k. .nd exploited 
propertie. of the rule., ,uch .. rule independence .nd .ub.umption to radically 
reduce the total number of model., Sleeman [1983]. 

Student, do make c.rel... .lip.,..»d it i. import.nt th.t the.e .hould not 
deflect the modeller unduly. The BUGGY project ...umed th.t if the .tudenf. 
response va. within « certain tolerance, that the error wa. a 
number-fact-retrieval problem and .hould be ignored, i.e., modelled as if the 
correct number-fact-retrieval had been made. LMS use. a aimple statistical 
procedure to return the model which it believes explains the students behavior on 
a set of tasks. 

2 ' 2 . : I — Friendly Natural Language Interface 

"Classical" Natural Language work, see Simmons {1970], provides parsers which 
are able to analyse sentences in natural language if the input is grammatically 
correct, which in turn implies it must be complete . On the other hand, humans 
regularly communicate with, and are understood when they use, incomplete and 
inconsistent utterances. Thus it is not surprising that ITSs have encountered a 
need to cope with these messier types of inputs. 

The parser which Burton [1976] implemented as part of the SOPHIE system was 
described as being a fu«y semantically-driven parser, and represented a pragmatic 
step in the .building of ITSs and in turn made an important contribution to Natural 
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l-ngu.ge proceeding. The par.er was .o named at i£ look . for auntie classes 
such a, measurement in the atudenfa input and not for a particular ayntactic 
entity, at in conventional parsers. It was designated iuz^ at it had the 
ability to ignore noi.e vorda - particularly a. it became .ore convinced that it 
was able to p.r.er a complete input. The ACE system, Sleeman and Hendley 11982], 
has extended this technique to deal additionally with inconai.tent and incomplete 
user explanations. 

Ill Another Perspective on the ITS and AI. 

Another perspective on the intercorrelation between AI and ITSs is provided 
if one thinks of the sub-systems which are needed in a complete ITS (Task 
Selector, Problem Solver, Presenter, Response Analyzer, Student Modeller and 
overall strategy critic) and then considers the important AI research areas which 
each of these subsystems raise. I have attempted one such listing: ' 

Problem Solver: 

Representntion, Search, Heuristics. 
Presenter /Analyzer: 

i 

Natural Language (incomplete and 
inconsistent input). 
Model ler: 

Inference, Representation, 
Consistency of data-bases, Dialogue 

Control. 

Critic: S 

h 

Inference, Representation. 
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It appear. th.t »n, of th. re.e.reh i..»e. currently of central concern to 
AI, .1,0 appear here, and w. have note, «,H„, in thi , eontMt th , 
need to be robu.t, u.er-tolerent and efficient. 

3. An evolving metho dology for buildin. ITS. 

The implementor. of en ITS frequently perform the following etepe: 

1. An.l y « protocol, for .indent. from the t.rg.t popul.tion .olving typical 
ta.k. .„d codify their difficultie./.i.under.t.nding.. (Thi. My UvoWe defiled 
"clinical" interview, with the .tudent.). 

2. Create dat.-ba... tor [he m ^ . ncludej f ^ ^ 
observed in step 2. 

3. Dm the ITS with .tudent. and in particular note .tudent error, which .re 
not spotted by the system. 

A. Carry out detailed student interviews to determine the nature of these 
misunderstandings; encode these as additional mal-rules. 

, Steps 3 and 4 .re repeated until the system captures the majority of the bugs 
which occur with the target population. 

3.1 Note s on the ITS Methodology 

IV The above emphasize, creating and debugging a data-base of rules which 
represent the student's misunderstanding of the domain. In fact. ITSs have a 
variety of data-bases, a* noted earlier, including domain knowledge, tutorial 
strategies (i.e., when to interrupt a student, how to present the essential 
diagnostic information etc.). Each of these data-bases needs to be articulated 
and debugged at indicated above. 

< j 

2. Articulating the .ev.r.l d.t.-ba.e. involved in an ITS i. an important 
contribution of thi. fi«Id to the theory of education. Such activitie. will 
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tr.n.fon. tr.ining „ d in.truction ham .n art for. to . .ci.nce . In other 
or... of hu»n .ctiwity. expert .y.te«, h.v. been the whicl., for eo.Bunie.ting 
knowledge within dLeipline. .. di.per.t. ..Medicine end Engineering, ,i. logy Md 
Aeeounting. (Thu. -any. people vie. ITS. a. . .„b-. c tivity of the Expert Sy.t... 
field: additionally having boild en expert »„y in.titution, ere beginning 

to e.k whether the. ,«e det.-b.ee c.n then be u.ed in in.truction.l .etting.. 
CUnce, * Let.inger [1981], in p.rticol.r, h.ve looked et the.. i..ue,). 
3. The methodology for building ITS! given earlier el.o bear, .triking 
.i«il.ritie. to the knowledge extr.ction-refine.ent cycle u.ed within the expert 
ey.tem'. p.r.digin, Haye.-Roth, Waterman & Lenat 11983]. 

3 ' 2 A Detailed lo <* gj rule ref i ne ment in the domain of introductory .1^. 

In 1981, I ran an experiment with 24 14-year-old algebra students to 
determine their competence at basic algebra tasks. Essentially this same 
data-base had been used earlier with 15-year-olds/and LMS had spotted a high 
percentage of their difficulties, sieeman 119*2]. The results of the experiment 
with the 14-year-olds were very, di fferent ; « high percentage ok the student's 
errors were not diagnosed by LMS. Again to verify that the difficulties noUced 
were not an artifact of interacting with LMS, I also admini stered' a few months 
later a paper-and-penci 1 test. I subsequently interviewed all those students who V " 
had significant problems on the second test and those who had had problems on the 
first test which appeared to be cleared up before the .second. 
The results of the interviews can be summarited as: " 
1. Some student, regularly solve algebraic equation, by searching for solutions.' 
Namely, substituting value, for the variable to determine if the equstion 
balance.. So given an equation 2 x X * 5 - 23, many student, would substitute X 
- 1, X - 2, X - 3, ... 
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2. Some students solve equations of the fern: 



M x X ♦ N x X - P 



if the 2X. are independent. I worked with student *ho. solved 9 t..k. in . 
row by . weird but consistent, el t orith», she solved ts.k, .„ ch N 
3 x X ♦ 2 U ■ 12 

ss 3 x 2 ♦ 2 ♦ A • 12 * . 

end wrote X * 2 end X * 4 
(The fir.t X eon.i.tently t . ken the v . lue Qf the tecon<J tomUulmt o( x> 
,he then copied down the ..cond coefficient - writing 3*2*2 thi. w.. then 
ev.lu.ted .nd the ..cond X w.. obt.ined by .ubtr.cting thi. f ro . the v.lu, on 
the RHS. For more di.cu..ion of thi. protocol ,ee Sleemen (in pre..J. 
3. Some .tudent. h.d . r.nge of eltern.tive method., ,„d confided they were 
unclear when to . P p ly which method. Some of the.e .tude.,t. occ.ion.lly u.ed the 
correct method to .olve en .Igebr.ic e,u.ti,n .nd .t other time, .ppeared to use 
. mel-ruU, .ee figure Z for ,x«,ple. of protocol, which include ^l-rule.. 
A. Consistent use of Mai-Rules 

Many of the .tudent. u.ed mel-rule. eon.i.tently. Ju.t over h.lf of the 24 
.tudent. we .aw mi.h.ndled precedence in equ.tion. of the form: 
2 ♦ 3 x X » 9 

A .ection for one .uch .tudent i. given in figure 2.1. Figure 2.II p.rt of . 
protocol produced by . ,tud*4 who collect. .11 the numeric ten., on the RHS of 
the e,u.tion irrespective of whether they .re "free" integer, or coefficient.. 

The .tudent, who cre.ted figure 2.III, ... rea.rk.bly con.i.tent with hi. 
«l-rul«, over . whole r.nge of t..k type.. Note in p.rticul.r ho. he h.ndled 
t..k c which involve. 3 X-t.rm.. H.ving worked t..k h, he noticed th.t when he 
moved the 4 «ro.. to the RHS he ch.nged the .ign .nd .o he .ugge.t.d th.t when he 
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"° VC X 2 x X) to to. LHS, he should also c*^ it, sign. H. 

then vcrbalitad that 

X - X it 0 

*° theli,S . 0 — «« «" !»£. and „ he rMli ,. d th , t tMi 

propo.ed .olution „.. „ 0 t po..ible. However. f or good «..„„ te ^ ^ _ 

i with the "revised" algorithm. 

.?4$ure 2 about here] 



5. "Saved Souls" 

During the on-lin. ..„ ion . ntoient consiitenely , olved ^ ^ 



form: 

3 x X« 4 x X - 13 at 
X ♦ X « 13 ♦ 3 + « 

However, during the .econd (pen-.„d- P .per te.t) end during the interview .he 
worxed the™ correctly. Moreover, when pre.ented with f,l. t i on . .Iterative, of 
the £or n given .hove, .he «.. ehle to .., clearly it w.. wrong .nd wa. able to 
explain clearly why it wa, wro,. S . Thi, behavior wa. noted with .ev.r.l 
students. 



3 * 3 Summary of the exp eriment- - 

The principal observations stages were: 
1. Some .tudent. have un.table "bug.", that i.^they have a whole range of 
"«thod,' -olving the ..*e ta.k, and they are unclear when to app!y w.ach 

-thod, sieeman |in preparation] for a .ore detailed di.cu'..ion cf thi..) 

•2. Schema for generating «l-rule.. Several .tudent. gave u. ..valuable in.ight 
into their "logic" when a.ked why the changed 
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2 x X ♦ 3 x X • 19 to 

X ♦ X • 19 - 2 - 3 

they Mid they were oerely collecting ell the Xs to one side and the numbert to 

the other. Thi- "ache.." enable u. to correctly, predict how the ..me students 

would process other tasks such as: 
5 x X - 2 x X ♦ 18 

3 « Differing types of student-errors 

Figure 3 shows the same task being solved differently/and both incorrectly by 

two separate, students. I have classified the error of figure 3.1 as a 
manipulative error, as I believe the student essentially knows the rule but has 
made an error in carrying it out. I have classified the error of figure 3.2 as a 
parsing error as I believe this represents a significant misunderstanding of 
algebraic notation. (Additional examples of manipulative and parsing errors are 
given in Figure 2). This distinction is born up experimentally. During the 
course of interviews the student whose protocol is given in figure 3.1 was able 
explain the various stages in the transformation, whereas the other student 
asserted he went from the first line to the second line in one step (i.e. there 
were no intermediary steps). 
{Figure 3 about here] 

4. The difficulties of teaching algebra have been greatly underestimated. The 
nature of the misunderstandings noted here are being investigated, to see if 
teaching sequences con be devised to avoid some of the misunderstandings noted 
with these students. 

5. Finally the rule set has been enhanced so that LMS would be able to handle 
many of the error, which it was previously unable to detect. (Because the parser 
errors have a different form from the manipulative mal-rules, LMS has also been 
somewhat enhanced). 
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For a discussion of current activities of this work see Sleeman [in press]. 

• , * 

4. Concluding comments 

The field of ITS currently has a major opportunity. It is now possible to 
run large ITSs on a personal computer where before one needed a sizeable fraction 
of large DEC-10, and so it is feasible to run large scale teaching experiments. 

However, many of the technical issues raised earlier remain. I predict that 
the focus of work in the next decade will be on: 
. analysis of the students' 
misunderstandings, 
providing more psychological 
explanations for the ma 1- rules 
observed, Brown & VanLehn 1 1980) and 
Sleeman [in preparation!. 
. providing more robust and more 

versatile natural language interfaces, 
implementing more robust user/ student 
modelling systems. 

To date these have been very limited in their scope and only captured the user's 
competence on a narrow task. To be really effective these models must include 
extensive information about the user, and be able to activate a body of inference 
rules which will enable the system to rapidly build a "crude" model from only 
general characteristics of the user. Examples of a general inference rule might 
be "if young and male assume the user is aggressive", an example of a more 
specific inference rule might be "if a second-year engineering student assume the 
user knows about thermodynamics". The mere specific inference rules should have a 
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higher credibility then the sore general rules. For • sort detailed discussion of 
user modelling see Rich (1983] end Sleeaan 1 1984 J. 

To those in e sub-field, progress often appears to be frustrit ingly slow. 
However, if one recslls that only two decedes, CAI itself wse ia its infancy, then 
one might well be satisfied with the progress with ITSs. Moreover, there is good 
reason to expect the rate of progress will accelerate now that we heve very such 
better programming environments in which to build systens and the possibility of 
using them in classrooms. 



15 



A. BaiT, M. Bearp* & R.C. Atkinson (1978). The computer as a tutorial laboratory: The Stanford BIP 
project, International Journal of Man-Machtiw Sttirito ffl ft PP 567-596. 

J.S. Brown & R.R. Burton (1978). Diagnostic models for procedural bugs In basic mathematical 
skills, in Cognitive Science. 2- pp1 55- 1 92. 

J.S. Brown, R.R. Burton & A.G. Bell (1975). SOPHIE. A step towards a reactive learning 
environment. International Journal of Man-machine Studies'. £, pp675-696. 

* 

J.S. Brown, R.R. Burton & J.deKleer (1982). Pedagogical, natural language and knowledge 
engineering techniques in SOPHIE 1,2 and a In intelligent Tutoring Systems , eds D. Sleeman & J ; S. 
Brown. Academic: London, pp227-282. 

J.S. Brown & K. VanLehn (1980). Repair Theory: a generative theory of bugs in procedural skills. 
Cognitive Science. 4, pp379-426. 

R.R. Burton (1976). Semantic Grammar: an engineering technique for constructing Natural 
Language understanding systems. BoJL EsmneK A Newman report. 3453 . • 

R.R. Burton & J.S. Brown (1982). An investigation of computer coaching for informal learning 
activities, in intelligent Tutoring Systems, eds D. Sleeman & J.S. Brown. Academic: London, 
PP79-98. ' ' 1 * 

J.R. Carbonell (1970). Al in CAI: An Artificial Intelligence Approach to Computer-aided instruction. 
IEEE Transactions on Man-Machine Systems, 11. pp.ign.9ra> 

W.J. Clancey a R.Letsinger (1981). NEOMYCIN: reconfiguring a rule-based expert system for 
application to teaching. Proceedings flf loL 4sM Conf. on AL Z, pp 829-838. 

W.J. Clancey (1982). Tutoring rules for guiding a case method dialogue. In Intelligent Tutoring 
Systems, eds D. Sleeman & J.S. Brown. Academic: London, pp201 -225. 

R.B. Davis, E. Jockusch & C. McKnight (1978). Cognitive processes in learning Algebra. Journal of 
Children's Mathftn^tir.a'f Behavior. 2,pp1-320 

M.R. Genesereth (1982). The role of plans In intelligent teaching systems. In Intelligent Tutoring 
Systems, eds D. Sleeman & J.S. Brown. Academic: London, pp137-155. 



■* ♦ ^ ■ „ 

* « . t 

t 

% 

1 

I.P. Goldstein (1982). The genetic graph: a representation for the evolution of. procedural 
knowledge. In Intelligent Tutoring Systems, eds D.SIeeman & J.S. Brown. Academic: London, 
PP51-77. 

J.R. Hartley & D.H. Sleeman (1973). Towards intelligent teaching systems, Int. J. of Man-Machine 
Studies, 5, pp.21 5-236. 

F. Hayes-Roth, D.A. Waterman & D.B. Lenat, eds (1983). Building Expert Svst»m^ Addison-Wesley: 
Reading. 

R. Kimball (1973). Self-Optimizing Computer- Assisted Tutoring: Theory and Practice. Stanford 
IMSSS Technical ration pQq % 

R. Kimball (1982). A self- improving tutor. In lnleJtai JjiloJiOfl Systems, eds D. SIeeman & J.S. 
Brown. Academic: London, pp283-307. 

M. Mate (1982). Towards a process model for high school algebra errors. In Intelligent Tutoring 
Systems, eds D. Sleeman & J.S. Brown. Academic: London, pp25-50. 

M.L. Miller (1982). A structured planning and debugging environment for elementary programming. * 
In Intelligent Tutoring Systems , eds D. Sleeman & J.S. Brown. Academic: London, ppH9-i35. 

T. O'Shea (1982). A self- improving quadratic tutor. In Intellige nt Tutoring Systems, eds D. Sleeman 
& J.S. Brown. Academic: London, pp309-336. 

' o 

E.A. Rich (1983). Users are individuals: individualizing user models. International Jflumaj oj Mflfli 
machine Studies. 18. pp199-214. 

R.F. Simmons (1970). Natural language question answering systems. CACM, 13. ppl5-30. 

D.H. Sleeman (1982). Assessing competence in basic algebra. In Intelligent Tutoring Systems, eds 
D. Sleeman & J,S t Brown, Academic: London, pp185-199. 

D.H. Sleeman & J.S. Brown (1982). Intelligent Tutoring Systems: an overview, in Intelligent 
Tutoring Systems, eds D. Sleeman & J.S. Brown, Academic: London, pp1 - 1 1 . 

D.H. Sleeman & RJ. Hendley (1982). ACE: A system which Analyses Complex Explanations. In 
intelligent Tutoring Systems, eds D. Sleeman £ J.S. Brown. Academic: London, pp99-1 18. 



9 

ERIC 



17 



D.H. Sleeman (1983). A rule directed modelllntf'ayateml 4n Machine Learning, eds R. Michalaki, 
J. Carbonell, & T. M. Mitchell, Tioga: Palo Alto, pp483-51 0. 

D. Sleeman (1 984). UMFE: User Modefllng Front-end subsystem. Stanford -HPP working paper. 

D. Sleeman (In press). Basic Algebra revisited: a study with 14 year-olds. Journal of Mathematical 
B eha vior- ■ ' 

D.H. Sleeman (in preparation). An attempt to understand students' understanding of Algebra. 

< * * 

R.L. Smith, H. Graves, t.H. Blaine & V.G. Marinov (1975). Computer-assisted axiomatic 
mathematics: informal rigor. In Computer Education O. Lecarne & R. Lewis (eds). North Holland: 
Amsterdam. 

A.L. Stevens & A. Collins (1977). The goal structure of a socratic tutor. fioJi Beranek & Newman 
Technical report 3518. 

A.Stevens, A.Collins & S.E. Goldin (1982). Misconceptions in student's understanding. In 
Intelligent Tutoring Systems, eds D. Sleeman & J.S. Brown. Academic:- London, pp13-24. 

A. Stevens & D. Gentner (1 983). M ental models Erlbaurn: New Jersey. 

P. Suppes (1967). Some theoretical models for mathematics learning. Journal pi Research and 
Development in Education. 1. pp5-22. . ^ . • 

* • 

L. Uhr (1969). Teaching machine pro-ams that generate problems as a function of interaction with 
students, proceedings oj Ihfi 241h National Conference . pp125-134. 

P.Woods & J.R. Hartley (1971). Some learning models for arithmetic tasks and their use in 
computer-based learning. British Journal of Educational Psychology. 41 , ppflfi.4fl 



18 



Figure 1 

Some of the Intelligent Tutoring Systems Implemented in the 1970* s. 

. ' / - ; • • - ... ' 

1. Symbolic integration, Kimball 11973 & 1982 J. 

2. Solving quadratic (by £ discovery Method), O'Shea {1982 J. 

3. Axiomatical ly-based mathematics, the EX CHECK system, Smith et. al. { 1975 J ♦ 
4* Electronic trouble-shooting, the several SOPHIE systems, Brown, Burton > Bell 
1 1975 J and Brown, Burton & de Kleer [1982). 

5. Interpretation of NMR spectra, the PSM-ffriR system, Sleeman & Hendley I 1982 ]. 

6. Socratic dialogue in geography and metereology, the WHY system, Stevens, & 
Collins 1 19 7 7 J . 

7. Medical diagnosis, the GUIDON System, Clancey [1982J. 

8. Informal gaming environment: the WEST system, Burton & Brown [1982 J, and , the 
WUMpyS aystem, Goldstein ( 1982 J. 

9. PrograniTplan debugging, the SPADE system, Miller [1982], 

10. Basic programming, the BIP system, Barr, Beard & Atkinson [1976]. • 

11. A consultancy system for users of MACSCMA, an algebraic manipulation system, 
Cenesereth [1982]. 



flgurt 2 

ThrM exaaples of very consistently uitd HAL-RULES. 

t 

> 

I) Pupil AB-li on task sot 7. 

ft) The task given was r 4 ♦ 2 • X ■ 16 

Pupil writes 1) 8X - 18 

2) X - 2.6868 

b) The task given was : 2 * 4 • X ■ ,14 

Pupil writes lj*6»X«14 

2) X - 2.333 

c) Tha task given was : 3 + 5 • X - 11 

Pupn writes 1) 8 • X - 11 

(and 1s told she can leave 1t in that form) 

d) The task given -was : 5 - 3 ♦ X - 11 

Pupil -writes - i) 2 * X -- 11 ' 

(and 1s told she can leave it 1n that form) 

II) Pupn AB-17 on task set 5 

a) The task given was : 2 • X ♦ 4 • X - 12 

< Pupil writes l)*X*X«12-2-4 

2) X 2 - 6 ' 

3) X - ROOT 6 

p) The task glveh was : I • X + 3 • X - 10 

Pupil writes 1) • X • X - 10 - 2 - 3 

2) X ♦* 2 - 5 
(and 1s told he can leave 1t 1n that form) 
c) The task given was : 2 • X - 3 • X ■ 10 

Pupil writes l)*X*X«10-2+3 

2) X •• 2 - 11 

(and 1s told he can leave 1t 1n that form) 



III) Pupil AB-18 on task sets 5, 6, 7 and 8. 

a) The task given was : 2 * X ♦ 3 • X - 10 

Pupil writes 1) 2 • X - 10-2-3 

2) 2 • X ■ 6 

3) X ■ 2.S 

b) The task given was : 3 • X + 5 * X ■ 24 

Pupil writes 1) X + X - 24- - 3 - 5 

2) 2 • X - 16 

3) X * 6 

c) The task given was' :3*X + 4*X + 5*X-24 

Pupil writes - 1) X ♦ X + X - 24.- 3-4-6 

2) 3 * X - 12 * 

3) X' • 4 

d) The task given was : 2 • X ♦ 4 - 20. 

V; Pujjn writes •, 1) X ■ 20 - 2 - 4 

2) X • 14 



BEST COPY AVAILABLE 

20 



P«flt 2 



• ) Tha task given wis : 3 • X <• 6 - -7 

Pupil writas 1) X ■ 7 - S - 8 

*) X » -1 

f ) Tha task fllvan was : 4 ♦ 3 • X - 14 

Pupn irltn • l) X ■ 14 - 3 - 4 

I) X • 7 

0) Tha task givan was : 5 + 8 • X « 20 

Pupil writas 1) X ■ 20- 5 - 8 

•' 2) X • 9 

h) Tha task glvan was \- 4 • X - 2 • X ♦ 8 



Pup 11 writas 



1) 2 • X - -4 ♦ 2 ♦ 6 

2) 2 * X - 4 

. 3) -X - 2 



Pupil than wrota 1JX-X-2+6-4 

2) 0 - 4 
and QUITS. 

1) Tha task glvan was : S*X-3*X#6 



Pupil writas 



1) 0 - 4 

and QUITS. 
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Figurt 3 

Two protocol! iliuit rating both Manipulative and parsing ul- rules. 

I 6 x X - 3 x X + 12 

9 x" 1 * 12 

X - 12/9 

X - 4/3 

XI 6 x X • 3 x X - 12 

X x X - 12 ♦ 3 - 6 

2 x X - 9 

X - 9/2 
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